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Oh 1 Abstract 

^3 \ We consider a queue with renewal arrivals and n exponential servers in the Halfin-Whitt 

heavy traffic regime, where n and the arrival rate increase without bound, so that a crit- 
ical loading condition holds. Server k serves at rate /Xfc, and the empirical distribution of 
{(J-k}k=i....,n is assumed to converge weakly. We show that very little information on the ser- 
vice rates is required for a routing mechanism to perform well. More precisely, we construct 
■ a routing mechanism that has access to a single sample from the service time distribution of 

each of n^ +e randomly selected servers (e > 0), but not to the actual values of the service 
rates, the performance of which is asymptotically as good as the best among mechanisms that 
have the complete information {fik}k=i,...,n- 

Keywords: Halfin-Whitt regime; routing policies; service time sampling 
MSC2000: Primary: 60F17. Secondary: 68M20, 90B15, 90B22, 60K30, 60K25. 

O 

1 Introduction 

X 

In the many-server parametric regime of Halfin and Whitt [10], a critically loaded diffusively 
scaled system has the property that the fraction of time when queues are empty is neither close 
to nor 1, a situation that is often observed in applications. Particularly, it has been suggested 
that this regime is suitable for modeling large call centers [8], and various models motivated by 
this application have been studied, where a many-server system operates in this regime (see [15] 
for a review). In models that involve heterogenous servers, a principal problem is to find an 
efficient routing policy [1, 2, 4, 5, 6, 9, 13, 14]. In all previous works on routing control in this 
regime, the proposed routing mechanisms are assumed to have complete information about the 
service rates of each server (where by 'rate' we refer to the parameter of the exponential service 
time distribution, assumed by most authors; however see [13] for more general service times). 
Since often in applications the routing control mechanism has little knowledge of the performance 
of each individual server, it is natural to ask whether it can perform near optimality with less 
information on these parameters. Our goal in this note is to argue that sufficient information for 
this purpose is a single sample of service time from a negligible fraction of the servers. 
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The pioneering work of Halfin and Whitt [10] considers a queue with renewal arrivals and 
identical exponential servers, where the number of servers and the rate of arrivals are scaled up so 
that the queue remains critically loaded. The second order asymptotics of the process representing 
the number of customers in the system is shown to converge to a diffusion. When the servers are 
heterogenous, it was shown in [1] that, in presence of customers of a single class, the policy which 
routes jobs to the fastest server among those that are free at the time of routing (and does not 
allow interruption of service) is asymptotically optimal in terms of the queue length as well as 
the delay of an arriving customer. Analogous results are available for the case of random, i.i.d. 
service rates [3] and, under appropriate assumptions, for hyper exponential service times [13]. 

We mention that works that characterize the fluid and diffusion scaling limits are available 
for homogeneous servers with general service time distributions [11, 12]. The question that we 
address here is also very natural in this wider context. Note, however, that for heterogenous 
servers with general service times, an asymptotically optimal routing policy is not known even 
when the routing mechanism has access to all service time distributions (with the exception of 
[13]). For this reason we confine our treatment to the exponential case. 

As mentioned above, we assume that the routing mechanism has access only to samples from 
the service time distribution of some of the servers. We show that, perhaps counter- intuitively, 
very little sampling is required for asymptotically optimal performance: It suffices to collect a 
single sample from each server in a set of r randomly selected servers, where r is as small as n^ +e 
(e > 0). The proposed policy always routes jobs to non-sampled servers if such are available, 
and otherwise, routes to the server for which the sampled service time is smallest among the 
(sampled) servers that are available at the time. It is shown to be asymptotically optimal in 
the sense that the diffusion limit of the process representing the total number of customers in 
the system (characterized in Theorem 2.1) is stochastically dominated by any subsequential limit 
under any (work conserving, nonanticipating) policy (see Theorem 2.2). This includes policies 
that have access to the complete information on service rates. A similar statement holds for the 
queue length processes (simply by (13)). 

A clear practical advantage of our approach is that it is not necessary to invest in measuring 
various characteristics precisely, or collect accurate information on the performance of the servers. 
In addition, the policy proposed has a desired robustness property in that its performance is 
nearly optimal regardless of the values of system parameters, as long as the basic assumptions 
hold. These assumptions on the empirical measure of the rates and its first and second-order 
limits (l)-(3) are quite general, and so are the assumptions on the limiting distribution. 

The proofs are based on an estimate on the number of errors in ordering the servers according 
to their sampled data (Lemma 3.1), an estimate on the total idle time encountered by servers that 
have relatively high priority (Lemma 3.2), and the technique developed in [3] (proof of Theorem 
2.1). 

In the next section we describe the model and the proposed policy, and state the main results. 
The proofs appear in Section 3. 

2 Model and main results 

We fix some notation. Denote by B the space of functions from R + to R that are right continuous 
on R + and have finite left limits on (0, oo) (RCLL), endowed with the usual Skorohod topology 
[7]. If X n , neN and X are processes with sample paths in B (respectively, real- valued random 
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variables) we write X n =4* X to denote weak convergence of the measures induced by X n on D 
(respectively, on R) to the measure induced by X, as n — > oo. For X G D we write := 
sup 0<s<t |X(s)|. For x G R, write x + = max{x,0} and x~ = max{— x,0}. 

A complete probability space (£2, P) is given, supporting all random variables and stochastic 
processes defined below. Expectation w.r.t. P is denoted by E. We consider a single queue fed by 
renewal arrivals, with parallel exponential servers. The model is parameterized by n G N, where 
n also represents the number of servers. The n servers are labeled as 1, . . . ,n, and, for the nth 
system, deterministic parameters /xjj G [fi, p] are given, where /xj? represents service rate of server 
k, and < fx < p, < oo are constants independent of n. We assume weak convergence of the 
empirical measure of {^}, 

L n = n- 1 J2^ k ^m, (1) 
k 

where m is a probability measure on R (supported on p]). The mean is denoted by (i = f xdm. 
A second order type approximation is further assumed on the rate parameters, namely that the 
limit 

n 

lim n~ 3 ^ (//£-//) :=£ (2) 
k=i 

exists as a finite number. Denoting /x* = essinf m, we finally assume 

lim : ^1 < /A* - e}n~^ =0, e > 0. (3) 

Example 2.1 A special case of assumptions (1), (2) and (3) is when there is a fixed number of 
pools of servers with a^n + O(l) servers at pool i, and where each server at pool i serves at rate 
bi + Cjn"2 (for constant aj,6j,Cj; > 0), a setting that is common (for example [1] in a single 
class setting, and [14] in a multiclass setting). 

Example 2.2 We point out that there is more flexibility in the choice of the parameters. For 
example, if we have two pools of size 0.2n + n* and 0.8n + n~> with rates 1 + 4n~6 + n~2 and, 
respectively, 2 — n~e 1 then our assumptions still hold. A more general case is as follows. We 
have a fixed number of pools of sizes a^n + /j(n), with respective rates bi + Cjii~2 + gi(n). Then 
assumptions (l)-(3) hold provided that fi(n) = o(n), gi(n) = o(l) and that the limit 

lim y^aigi(n) 

i 

exists. This is verified by a straightforward, if lengthy calculation using /j, = J2 i aibi. 

Example 2.3 It is sometimes very natural to regard the rates {fik} as random variables, and thus 
to consider the queueing process, as well as its scaling limit, as processes in random environment. 
The case where the service rates are i.i.d. random variables, drawn from a common distribution 
m, was considered in [3]. In this case, the law of large numbers implies that (1) and (3) hold with 
probability one, and the central limit theorem implies a variation of (2), in which ju is a normal 
random variable. Although we assume throughout that the service rates are deterministic, we 
would like to comment that all our results can be formulated for an i.i.d. random environment, 
with basically the same proofs. 
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The initial configuration is now described. Let Qq be a Z + -valued random variable, representing 
the initial number of customers in the buffer. Let B% Q , k = 1, ...,n be {0, l}-valued random 
variables representing the initial state of each server, where B% Q = 1 if and only if server k 
initially serves a customer. We restrict to non-idling policies, so that in particular Qq > only 
if = 1 for all k = 1, . . . , n. The total number of customers initially in the system is denoted 
by Xq = Qq + Y?k=i ^ko- Note that, by assumption, we have the relation Qq = (Xft - n) + . We 
assume 

X£:=n-!(X "-n)^£o, (4) 

where £o is a random variable. 

To define the arrival process, we are given parameters \ n > 0, n £ N satisfying lim n A™/n = 
A > 0, and a sequence of strictly positive i.i.d. random variables {U(l),l G N}, with mean 
EU (1) = 1 and variance C 2 = Var(£/(1)) € [0, oo). With Y!i = 0, the number of arrivals up to 
time t for the nth system is given by A n {t) = sup{Z > : J2i=i U(i)/\ n < t}. The arrival rates 
are further assumed to satisfy the second order relation 

limrW(A n - nX) = A, (5) 

n 

for some Ael. The 'heavy traffic' condition on the first order parameters is assumed, namely 

A = fi, (6) 

indicating that the system is critically loaded. For each k = 1, . . . , n, we let B"^ be a stochastic 
process taking values in {0,1}, representing the status of server k: when B^(t) = 1 [resp., 0] 
we say that server k is busy [resp., idle]. Let I^{t) = 1 — B%(t) for k = 1, . . . ,n, and t > 0. 
For k = 1, . . . , n, let [resp., D^] be a Z + -valued process with nondecreasing right-continuous 
sample paths, representing the number of routings of customers to server k within [0, t] [resp., the 
number of jobs completed by server k by time t]. Thus 

BUt) = B% + R n k (t)-D%(t), fc = l,...,n, t>0. (7) 

To describe the processes D^, let {Sk,k £ N} be i.i.d. standard Poisson processes, each having 
right-continuous sample paths. The processes are assumed to satisfy 

£>£(*) = SkpW)), k = l,...,n, (8) 



where 

1k(t)=(% f B n k (s)ds, fc = l,...,n. (9) 
J o 

Let X n , Q n and / n be defined as 

n n n 

x"(t)=xz+A n ( t )-j2mt), Q n (t) = Q%+A n (t)-Y,R n k(t), i n (t) = Y,im- m 

k=l k=l k=l 

These processes represent the number of customers in the system, the number of customers in the 
buffer and, respectively, the number of servers that are idle. 
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The routing policy, that will be described below, does not have access to the service rates [1%, 
but it has access to samples from the service time of r of the servers, selected at random, and 
no information at all on service rates of the others. More precisely, let r = r n <G N, r < n be 
given, and let E = E n be a random variable uniformly distributed over the set of all subsets of 
{1, . . . , n} that have cardinality r. We denote E c = {1, . . . , n} \ E. For each k £ E, let = cr r k l 
be an independent copy drawn from the service time distribution of server k. That is, Ok is an 
exponential random variable with parameter ^ and, conditioned on E, {crakes are independent. 
We choose 

r n = [rA], (11) 

where (3 r 6 (\, 1]. Denote flk = l/o~k, k E E. 

The four stochastic primitives introduced, as listed below, are assumed to be mutually inde- 
pendent, for each n: 

{ x o > { B k,o}k=i,...,n) , {Sk}keN, A n , (^E n , {a%} keS ^j. (12) 

Routing is based on an ordering of the servers according to whether they are in E and, within 
E, according to the value of A permutation Rank = Rank n of {1, . . . , n} is defined as follows. 
On the probability-one event that the jl^ are all distinct, the set E is mapped by Rank onto 
{1, . . . , r} (and E c onto {r + 1, . . . , n}). For k,l G E, Rank(/c) < Rank(Z) if and only if jik < V-l- 
For k, I £ E c , Rank(/c) < Rank(/) if and only if k < I. 

The routing policy favors servers ranked higher (namely those that have high value under the 
map Rank). That is, when a customer arrives to the system to find more than one idle server, the 
customer is routed to the server with highest rank among those servers. Since it is assumed that 
the routing policy is work conserving (non- idling) , when the queue is nonempty and a server has 
just finished serving, a customer (from the head of the line) is routed to this server, and when a 
customer arrives to the system to find exactly one server that is idle, it is instantaneously routed 
to that server. As a result, 

Q n (t) = (X n (t) - n)+ I n (t) = (X n (t) - n)~ (13) 

holds for all t. Also, service is non-interruptible, in the sense that a customer completes service 
at the server it is first assigned. 

This completes the description of the process 

7T " := ({BU,{Rk},{m,X n ,Q n ,n- 

It can be seen that this description uniquely determines IIq . We sometimes refer to this process 
as policy IIq. Later we use some of the symbols above (such as X n ) to denote quantities that 
have the same meaning (such as the number of customers in the nth system) under a different 
routing policy 77™. To avoid confusion, we therefore make specific reference to policy 77q when 
necessary. 

Finally, we make a simplifying assumption about the initial occupation of servers, namely that 
only servers that are ranked low may initially be idle: 

B k,0 = 1 {Rank(fc)>/J} ) (14) 
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where 

= (X5 - n)- (15) 

is the initial number of idle servers. 

Let X n be a centered, normalized version of the process X n , defined by 

X n = n-3(X n -n). (16) 

Our main result is the following. 

Theorem 2.1 Under policy IIq, the processes X n converge weakly to the unique solution £ of 

£(t) = £o + + # + / *>°> ( 17 ) 

■/ o 

where a 2 = [i(C 2 + I), (3 = \ — ji, and w is a standard Brownian motion, independent o/£o- 

The result above is to be compared with Proposition 4.2 of [1] and Theorem 2.2 of [3] (for 
the case of a finite number of server pools and, respectively, random environment). In these 
references, equation (17) arises in the limit under a policy defined similarly to 77 , but where the 
servers are ordered according to the actual values of k = 1, . . . , n. In [1] it is further shown 
that this policy asymptotically achieves the best performance in a large class of routing policies. 
Because our setting is different than [1], we will state and prove an analogous result, so as to 
exhibit that ilo is asymptotically optimal. 

Toward this end, let us first comment on an alternative representation of the departure process. 
By (8), this process is given as J2k=i = Ylk=i Sk(T£(t)), where Sk are independent standard 

Poisson processes. In fact, the departure process can also be represented as 

n n 

Y,D n k (t)=S n (Y,^(t)), (18) 

k=l k=l 

where, for every n, S n is a standard Poisson process, independent of the remaining primitive data, 
that is, of the first, third and fourth items of (12). This statement (along with a variation of it, 
stated in Section 3) is due to a standard superposition argument for Poisson processes, for which 
the reader is referred to Proposition 3.1 of [3]. 

We now define a class of policies by keeping the description of this section but abandoning 
the specifics of the routing mechanism. More precisely, we write 77 ra G V n for any process 

n n = ({Bn,{R n kh{m,x n ,Q n ,n 

satisfying all relations stated throughout this section, from its beginning to the statement of 
Theorem 2.1, save the two paragraphs following display (12), and satisfying, in addition, work 
conservation (13) and the representation (18), for some standard Poisson processes S n , indepen- 
dent of the remaining primitive data. Note, in particular, that the routing mechanism may have 
access to {fJ-k}- See Remark 3.2 about the role played by the work conservation condition (13). 
We refer to any element of V n as a policy. 
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Theorem 2.2 For n G N and any policy 77™ G V n , let X n be the normalized version (16) of the 
corresponding process X n . Then there exist processes E n that converge weakly, as n — > oo, to the 
solution £ to (17) and 

X n {t) > ~ n (t), t > 0, P-a.s., n G N. 

Since by Theorem 2.1, £ is obtained at the limit under 77q , the result above demonstrates that 
TTq asymptotically optimal. 



3 Proofs 

We begin with the following. 

Lemma 3.1 Let < <fi < ip < oo, (3 > \, c\ > and C2 > be given constants. For n G N denote 
I 1 = l\ = [cin' 3 ], £ 2 = £^ = fenP], and let 0™, ...,</>" and Vi\ ■■■ be positive real numbers 
with 

sup <j>? <<f><ijj <mf^. 

For n G N and z G {1, ...,£} let <Pf [resp., ^f] be an exponential random variable with parameter 
[resp., ip™]- For each n assume that are mutually independent and that so are {^ n }. 

Then there exist 7 > and k > such that, with 6 n = 7 log n, one has 



}™ p EVr>M^ l+K = ' 



(19) 



lim P \J2 1 {9 



1=1 



{*?>*»} 



> n2" 



(20) 



Proof. By stochastic monotonicity of the exponential random variable with respect to its param- 
eter, it clearly suffices to prove the claim for the case = (j), ipf = ip, all n and i. To prove the 
claim under this assumption, let k and 7 be strictly positive constants satisfying 



Write <Pj [resp. ^] for ^ n [resp., Wf 1 ]. Then for any a > and {6 n }, 



(21) 



1 -n 



I <e a ^ +, £exp 



1=1 



1 

«n2 



+« 



e— (P(xi < 9 n ) + e- Q P(xi > n ))*" 

= e anh+K exp {l\ log (l + e-^» (e" a - 1)) } 
<exp{an5+ K -^(e^(l-e" a )}) . 



7 



For the last expression to converge to zero, we need K n := ari2 +K — 1\ (e ^ n (l — e ")) — > — oo. 
Fix v > and set 9 n = 7 log n, a = a n = v log n. Then 

K n < z/logn • n5 +K - (cm? - 1) (V^l - n'^ . 

Since by (21) \ + k < /3 — ^7, we have if n — > —00, as desired, and thus (19) holds. 
Fix 77 > 0. Since (1 + x) k < e kx for x > —1, we have 



P Eta > n*"" < e-^^^exp 



i=i 



^E 1 

i=l 



{&i>e n } 



3-" ^ 



1 - e - ^" + e^e - ^" 



exp 



{«(« 



0} 



exp 



{-7777,2 K 



(e""-l)}, 



where on the last line above we substituted 6>„ = 7logn. The expression on the last line converges 
to zero because \ — k > [3 — tpj by (21), and (20) follows. ■ 

Remark 3.1 (a) The convergence in (19), (20) is at a geometric rate, as the proof shows. Thus, 
by the Borel-Cantelli lemma, both events occur for only a finite number of n, with probability 
one. 

(b) As can be seen in the proof, k and 7 depend only on (3, 4> and tp (cf. (21)), and not on {a}. 

Fix e > and let a G (/x*, /it* + e) be a continuity point ofn-> m([0, x]). In what follows, the 
symbols n and e are omitted from the notation of all random variables and stochastic processes, 
and from the parameters /i^. Let Mq = [fx, — e), M\ = [/j* — e, a) and M2 = [01, A*] (where [a, 6) 
and [a, 6] are interpreted as the empty set if a > b), and set 



Denote 



Ki = {k€{l,...,n}:fi k €Mi}, i = 0,1,2. 

/W(t)= E m r%)=E^), * = 0,1,2. 

Let also /W = n _ 3j«. By (8) and (10), 

n 

X(t) = X + n~lA(t) - n"3 £ 5 fc (T fc (*)). 



(22) 



(23) 



fc=i 



By a superposition argument for Poisson processes (cf. Proposition 3.1 of [3]), 



X(t) = X + n-*A(t) -n-*^S®(T®(t)), 

i=0 



(24) 
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where i = 0,1,2 are standard Poisson processes, mutually independent, and independent of 
the first, third and fourth items of (12). In particular, 

D®(t):=^2D k {t) = S®{TW(t)), i = 0,1,2. (25) 

keKi 

A calculation based on (6) and (24) shows (see a detailed derivation at the end of this section) 

X(t) = X + W(t)+bt + F(t), (26) 
where we recall that all quantities depend on n and e, and where 

2 

W{t) = A{t)-Y J W^{t), (27) 

i=0 

A{t) = n~*(A{t) - \ n t), (28) 
W {{) {t) = n^{S {i) {T^{t))-T {i \t)), i = Q,l,2, (29) 



n 



b = n -^(X n - nA) - n - 5 J^(/i fc - n), (30) 

fc=i 

F(t) =n-2 / Vft4(s)d S . (31) 
7o fc=l 

Lemma 3.2 Under Tig , given t > and e > 0, 

1^1 *,t - * in probability, as n — > oo. (32) 

Proof. Step 1: We will show here that there is a (deterministic) sequence a n increasing to infinity, 
so that a n n2 < r n , and such that, out of the a n n^ servers ranked lowest, the number of those 
that are in K 2 is 0(712), in the following sense: 

#{/c G K 2 : Rank(fc) < q»n^} ^ Q g 



1. 

n2 



We will apply Lemma 3.1. To this end let <fi £ (ft, a) be a continuity point of x 1— > m([/x, a;]). 

Let V = Let K = {I; £ {l,...,n} : ft < 0}. Since m([ft,0]) > 0, it follows^ from (1) that, 
for some constant c > and with probability increasing to 1, the cardinality of K is at least cn. 
Since the subset E is uniformly distributed and the number of samples satisfies (11), it follows 
that, on some events fi n satisfying P(Q n ) — > 1, 

for a constant ci > 0. Recall ft = l/cr^, the reciprocal to the sampled service time. We apply 
Lemma 3.1 with <£>i being the samples with index set K D E, and <Pj those with index set 
K2 fl E. We obtain, that on an event i?" C 17™, which also satisfies lim n _>oo -P(i2") = 1, 

#{i6^nr:ft<l/y>>, (34) 
e tf 2 n E : ft < l/9 n } < n\- K , (35) 
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where, without loss of generality, ^ < ^ + k < j3 r . Now, (34) and the way the map Rank is 
defined, imply that all servers k with Rank(/c) < n^ +K have flk < l/0 n and are in U. As a result, 

#{k e : Rank(A;) < n^ +K } = #{k € K 2 D £ : Rank(A;) < r^+ K } 

< e k 2 n r : < 1/04 

by (35). This proves (33) with a n = n K . 
Step 2: Denote 

Jf' = {fcG{l,...,n}: Rank(/c) > a n n2 }, 

and I' = YlkeK' ^fc> T' = J2keK' Tk-> D' = YlkeK' ^k- An argument as the one following equation 
(23) shows that S'(T'(t)) = D'(t), t > 0, where S' is a standard Poisson process. Set S'(t) = 
n _ 2(S"(nt) — nt) and I' = n~^I' . We shall show that 

\I |* j — > in probability, as n — > oo. (36) 

Note first that the probability of the event rj\ := {/'(0) = 0} converges to one as n — ► oo. 
Indeed, by (14), B^q = 1 for all k with Rank(fc) > Iq. By (4) and (15), Iq < a n n~z with probability 
converging to 1 as n — > oo. Thus, with probability converging to 1, all servers k € K' are initially 
busy, namely P(r]i) — > 1 as n — ► oo. Let 

5(t) = ra _ 3 (S(nt) -nt), t>0, (37) 

where S is a standard Poisson process. It is well known (cf. Lemmas 2 and 4(i) of [4]) that both A 
(of (28)) and S converge weakly to a zero mean Brownian motion with diffusion coefficient A 2 C, 
and respectively, 1. 

Given 7 > 0, consider the event n := {|i"'|*t > 27712}. On the event 77 fl r\\ one can find 
< s < t < t such that I'(y) > for all y £ [s,t], and I'(t) — I'(s) > 7712. Since the servers in 
K' are all ranked higher than those in the complement set, the routing policy assigns all arrivals 
within [s,t] to K' servers. Hence by (7), (8) and using B\. = 1 — 1^, we have 

7 7i5 < j'(t) - I'( s ) = D'{t) - D'(s) - A(t) + A(s), 

and therefore 
7 < S'in^T'it)) - S'in^T'is)) - A(t) + A(s) 

B k {y)dy - \n*{t - s) - n"5(A n - A)(t - a). 

KfciV 

We have by (9) and (22) that ra -1 T( 2 )(i) < /it =: r. Also, by (5), the last term above is bounded 
by c(t — s) for some constant c independent of n and e. Let 

w T (x,z) = sup \x(s) — x(t)\, z > 0, 

|s-t|<z;s,t€[0,T] 
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denotes the modulus of continuity for x : [0, r] — > R. Define C(n, e) = n 1 J^fceA"' ^fc ~~ ^- Then 
on the event rjDrji, with <5 = t — s, we have 

7 < tD T (S', 2/25) + idt(A, 5) + n^C(n, e)5 + c<5. (38) 

By (1), (2), (6) and the definition of K' , 

n*C(n,e) <c\ — n~* ^ fi k < c\ - \m n < -c 2 a n , 

fc:Rank(fc)<n^a„ 

for constants c±, c 2 > and sufficiently large n. Hence 

P(\T'Lt > 2 7 ) = P(v) < Pi(n, e, 7) + P2(n, e, 7) + P(vi), 



where 



Note that 



pi(n,£,j) = P(there exists 5 G (0, a n 2 ) such that (38) holds), 

_ 1 

p 2 (n,e,^f) = P(there exists S G [a n 2 , i] such that (38) holds). 



Pi (n, e, 7) < P(«) T (5', 2/Za n 5 ) + % (A, a„ 5 ) > 7/2) , 

^ ^ _ 1 

p 2 {n,e,j) < P(w T (S',2p,i) + Wt(A,t) > -ct + c 2 a^). 

Since S' and ^4 converge to processes with continuous sample paths, both expressions converge to 
zero as n — > 00. Since lim n P(r/f ) = and 7 > is arbitrary, (36) follows. 
Step 3: Since K 2 C K' U {{K'f n if 2 ), we have 

By Step 1 (display (33)), the last term on the above display converges to zero in probability. Thus 
by Step 2 (display (36)), statement (32) follows. This completes the proof of the lemma. ■ 

Proof of Theorem 2.1. Based on Lemmas 3.1 and 3.2, the proof is similar to that of Theorem 2.2 
of [3] (only slightly simpler). We include it for completeness and because the proof of Theorem 
2.2 is based on it. By (26) and (31), one has 

X(t) = X + W(t) +bt + n* f X(s)-ds + e(t), (39) 

Jo 

(where all the above quantities depend on n) and, with I k = I k , 

e (t) = Y](A*fc - M*) / h(s)ds. (40) 
fc=i Jo 

Fix i > 0. By (2), (5) and (30), b -> (3 = \-% We show that the random variables {\W {i) ^j, i = 
0, 1,2, n G N} are tight. By (22) and (9), for i = 0, 1, 2, 

n^T^t) = n" 1 J] - n- 1 £ Mfe /" I k (s)ds. (41) 
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Hence < n,- 1 T^(t) < fit for t < t and all n. Thus by (29), |W W |* )t - < where 
S^\t) = n~2 (S( l \nt) —nt). Recall from the proof of Lemma 3.2 that converge to a Brownian 
motion. Hence |WW|„, f are tight. 

Next, note that \e(t)\ < jl J Q * \X(s)\ds. Thus the boundedness of 6, the tightness of the random 
variables Xq, |W"W| j and n G N (as follows from the convergence of A), and an applica- 

tion of GronwalPs lemma on (39), by which \X\*t < (|Ao| + |W|*t + \b\t) exp(2/2t), imply that 
{l-X^f, n G N} are tight. Since by (13), I = X~ , we have that {|/|*,f, n G N} are tight. 

The supremum over t < i of the absolute value of the last term in (41) converges to zero in 
probability, since p^ are assumed to be bounded and are tight. Also, since a is a continuity 
point of x ' ^ m([0, x]), we have that 

n _1 fj,k — > / xc/m =: pi, z = 0, 1, 2. 

Note that p = 0. As a result, n _1 (T(°),T^),T^)) -> p in probability, uniformly on [0j\, 
where p(i) = (0,p 1 t,p 2 t). Recall that (1, S^) are mutually independent, and that 

[resp., A] converges to a standard Brownian motion [a zero mean Brownian motion with diffusion 
coefficient X^C] (see comment following (37)). Thus (27), (29) and the lemma on random change 
of time [7, p. 151] show that W converges weakly to aw, in the uniform topology on [0, t\, where 
w is a standard Brownian motion and a 2 = XC 2 + pi + P2 = XC 2 + p = p(C 2 + 1). 

By the Skorohod representation theorem, we can assume without loss of generality that the 
random variables Xq and £o and the processes W and w are realized in such a way that, P-a.s., 

(X , W) -► (£ , ew) as n -> oo. (42) 

Let £ be the unique strong solution to equation (17). Then by (17), (39), the inequality \x~ — y~\ < 
\x — y\, and Gronwall's inequality, 

\X - eU, t - < (\X - Co| + \b-(3\ + \W- awlj+\el,t) exp(p*i). (43) 
Now, by (40), for n sufficiently large, 

|e| M - < si\IU ; t + ni\I {2) \*,i+ (ji* (44) 
By (3), the last term above converges weakly to 0. Combining (32), (42), (43) and (44), 
limsupP(|A — £ > £3) < limsupP(ce|/| i|Ij f > ea), 

n n 

where c G (0, oo) is a constant independent of n and e. Note that the law of |/|* j does not depend 
on £. Hence by tightness of n G N} the r.h.s. in the above display converges to zero as 

e — > 0. Thus |A — £1*^ — > in probability. Since i is arbitrary, we have A £. ■ 

Proof of Theorem 2.2. By (3) there exists a sequence 5 n > tending to zero such that Cn : = 
#{k : p£ < p* - 5„}n~2 — > 0. Note that (39), (40) still hold. Define Si as the solution to 

Si(t) = X + W(t) + fa + p* / Si(s)-ds. (45) 

■/ o 



12 



Then by (39), A := X — E-y is differ entiable, and, using the inequality a — b < — (a — b) + for 
a, b <G M, we have Z\(0) = 0, and 

| 4(( )>-„, 4(t )+ + | e(t ). 
Since Ik < n~? for each k, we have by (40) 

^e(i) > -u n , f n := 5 n \T\^t + /i*Cn, 

and A(0) = 0. By comparison with the ordinary differential equation du/dt = —fi*u + —v n , u(0) = 
0, we obtain that A(t) > —v n t, t < i. Hence X(t) > S(t), where we define E(t) = 5i(t) — v n t, 
t < i. 

It thus remains to show that E => ^. For this let us review the proof of Theorem 2.1. Rather 
than three processes (25) and correspondingly i = 0, 1,2, (29), we now have a single 

process D = Ylk-Dk given in terms of a single standard Poisson process S n (cf. (18)). The 
adaptation of relation (29) to a single process W is obvious. The arguments in the proof of 
Theorem 2.1 that lead to the tightness of and the convergence of W to aw hold with obvious 
modifications. As in that proof, we deduce that (42) can be assumed without loss of generality. 
Equations (17), (45) and Gronwall's inequality thus yield 

I Si - e|*,f < (|*o - Col + \b - P\ + | W - awUj) exp(^f). 

Hence (42) and the convergence of b to (3 imply that S± converges in probability to £ uniformly 
over [0, t]. The random variables f n converge to zero by tightness of Since i is arbitrary, we 

thus obtain that S => £. This completes the proof of the theorem. ■ 

Remark 3.2 Note that the non-idling property is used in the proof of Theorem 2.1 (on which 
the above proof is based) for deducing tightness of \I\* t t from that of l-X"!*^. As can be easily seen, 
\I\* t i are n °t in general tight if the restriction to non-idling policies is removed. 

Derivation of equation (26). Recall that n is omitted from some of the notation. By (24), (28)- 
(29), 



x( t ) = x + A(t) - w(i) (<) + n ~ 1/2 \ xnt - Yl T(i) (*)' 

i=0 i=0 
n ,.f 

= X + W(t) + n-^[\ n t-Y,Vk B k (s)ds\, 

k=i ^° 

where (27), (9) and (22) are used in the second equality. Since Bj, = 1 — Ij,, 

n ft 
X(t) = X + W(t) + n-s [A" - Y, Mfc] * + ^"^ 2 Mfc / 4(s)ds- 

fe=i fc ^° 

By (6), /x = A, hence by (30) the penultimate term above is equal to bt. This shows (26). ■ 
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